The present invention relates to an image processing device and an image processing method, and is particularly characterized by its thinning-out means used in image resolution conversion.
Recently, digital-imaging/digital-video equipment, such as a digital still-video camera and DVD, and especially multi-media related equipment among them is rapidly springing into wide use. Also in the field of display devices, dot-matrix displays, such as liquid crystal displays and plasma displays, are becoming widespread. Then, when a number of pixels of image data differs from a number of pixels of a device for displaying it, image processing for adjusting the number of pixels of the image data to those of the display device, i.e. resolution conversion of the image data, is required.
In order to display pictures in a system in which conventional display device having its own number of pixels display picture signal after thinning out some pixels, the method disclosed in Japanese Patent Application Publication No. H05-276436, a technique of detecting correlation between pixels and determining pixels to be thinned out, mean value reducing processing, or the like, has been used. Conventional thinning-out techniques in image processing are described below with reference to the block diagrams shown in FIGS. 5 and 6.
In FIG. 5, delay circuit 51 delays input signal and feeds the delayed pixels into image discriminating circuit 52. Image discriminating circuit 52 detects a correlation of difference between adjacent target pixels, and determines pixels with a smaller difference therebetween as pixels to be thinned out. It also determines if the pixels are a part of characters or a part of a natural image. When the pixels have a correlation of a small difference in gray level, they are determined as a part of a natural image and fed into linear interpolation circuit 53. When the pixels have a correlation of a large difference in gray level, they are determined as a part of characters and fed into adaptive thinning-out circuit 54. The signal thinned out by both processing circuits are reorganized by image reorganizing circuit 55 as an image and sent to respective pixels in a display device and displayed there.
FIG. 6 is a block diagram of a system using mean value reducing processing. Delay circuit 61 delays input signal and feeds the delayed pixels into operation circuit 62. Operation circuit 62 averages the pixel to be thinned out and the pixel adjacent thereto, extracts the pixel at a timing of a thinning-out pulse, and feeds the signal obtained after the thinning-out processing into image reorganizing circuit 63. Image reorganizing circuit 63 reorganizes the picture signal obtained after the thinning-out processing as an image and supplies the data to a display device at a timing corresponding to each pixel in the display device. The signal supplied from image reorganizing circuit 63 are sent to the respective pixels in the display device and displayed there.
FIGS. 7A and 7B show a case where a black-and-white image based on an amount of unit information (pixels) and including patterns of 8 lines and 8 columns is input. They show 8xc3x978 pixels before the thinning-out processing (FIG. 7A), and 6xc3x976 pixels obtained after the system shown in FIG. 6 has performed the thinning-out processing (FIG. 7B). In the 8xc3x978 pixels, those in columns b, c and f, g, and those in lines b, c and f, g are a pixel to be thinned out and a pixel adjacent thereto, respectively, and pixels B and F are produced from the pixels to be thinned out and the pixels adjacent thereto by the mean value reducing processing. In this drawing, display xe2x80x9cHxe2x80x9d can be read as xe2x80x9cHxe2x80x9d after the processing. Although the mean value reducing processing has caused no lack of information, it has produced gradation in the part on which it has been performed. It is because the reducing processing has reduced the image by simply adding up the pixel to be thinned out and the pixel adjacent thereto, i.e. those in columns b, c and f, g, and those in lines b, c and f. g, and diving the sum into equal halves to produce converted pixels.
FIGS. 8A and 8B show 4xc3x974 pixels before the thinning-out processing (FIG. 8A) and 3xc3x973 pixels obtained after the system using the mean value reducing processing shown in FIG. 6 has performed thinning-out processing (FIG. 8B). In the case shown in FIGS. 8A and 8B, the pixels to be thinned out and the pixels adjacent thereto, i.e. those in columns b and c and those in lines b and c, respectively, are composed of black and white. The mean value reducing processing cannot represent the original black line and produces an indistinct gray line; thus making the reorganized image more blurred than its original. With this processing technique, the reorganized line is more blurred than that in FIGS. 4A and 4B obtained by the processing method of the present invention described afterwards. With this processing method, lines and texts shown on a screen generally supplied by a computer are quite blurred.
Next, a case where a conventional thinning-out technique is used for a natural image is described. In FIGS. 9A and 9B, thinning-out processing is performed on input picture signal of a natural image by the system using the mean value reducing processing shown in FIG. 6. FIGS. 9A and 9B show input picture signal of a natural image, or the like, with gradation before the thinning-out processing that consist of input signal a, b, c, d, e, f, g, h, i, j, and k arranged in order, and a signal waveform that consists of signal a, B, d, e, f, G, i, j, and k and is obtained after the system using the mean value reducing processing shown in FIG. 6 has reduced five pixels (FIG. 9A) into four pixels (FIG. 9B). The numerical values are given as a guide of signal levels. In FIGS. 9A and 9B, when the mean value reducing processing is performed on pixel b to be thinned out and pixel c adjacent thereto, thinned out pixel B is produced with a signal level of 15 according to (10+20)/2. This thinning-out processing is performed on a portion of the stepped input signal that has a small variation in brightness. The linearity has slightly been lost; however, it is not such a level that causes a problem in the output signal obtained after the processing. Subsequently, when the mean value reducing processing is performed on a pixel g to be thinned out and a pixel h adjacent thereto, thinned out pixel G is produced with a signal level of 50 according to (60+40)/2. This thinning-out processing is performed on the peak of the stepped input signal and the peak of this input signal has been lost. That is, the output waveform after the processing has no edge, thus giving a blurred impression. Therefore, the final image becomes indistinct. This phenomenon is more pronounced as the thinning-out ratio is larger. In the above description, the thinning-out processing in the horizontal direction is described as an example, and in the vertical direction also, similar output signal can be obtained after the processing with completely the same operation, except that the direction in which pixels are thinned out is different.
However, when a natural image is thinned out using the conventional thinning-out processing shown in FIG. 5, image discriminating circuit 52 may make a mistake. If it determines a natural image as a part of characters by mistake, straight and curved lines outlining an object are output as irregular lines; thus a more unnatural image than its original is displayed. When the thinning-out processing is performed on an image composed of characters like texts displayed on a personal computer, the same seen in a natural image holds true. In addition, the conventional system has a problem that the circuitry for thinning out characters, and the like, is complicated, and moreover, a large amount of operations are required for thinning-out processing because thinning out method must be changed according to discrimination result of a natural image or characters.
In addition, the conventional mean value reducing processing shown in FIG. 6 has a problem of generating a blurred image in which gradation is produced in a part having continuous lines, such as characters and graphics.
An image processing device of the present invention includes: thinning-out instruction means for instructing a pixel to be thinned out; coefficient determining means for detecting correlation between the pixel to be thinned out instructed by the thinning-out instruction means and a pixel adjacent to the pixel to be thinned out, and for determining a coefficient according to the detected correlation; and operation means for producing corrected pixel data using a predetermined operation formula, based on data of the pixel to be thinned out, data of the pixel adjacent to the pixel to be thinned out, and the coefficient determined by the coefficient determining means.
An image processing method of the present invention includes the steps of: instructing a pixel to be thinned out; detecting correlation between the pixel to be thinned out and a pixel adjacent to the pixel to be thinned out; determining coefficients according to the detected correlation; and producing corrected pixel data using a predetermined operation formula according to data of the pixel to be thinned out, data of the pixel adjacent to the pixel to be thinned out, and the determined coefficients.
The image processing device of the present invention determines a pixel B produced by the pixel to be thinned out and the pixel adjacent thereto in a horizontal direction or a vertical direction, in a 4-pixels block consisting of pixels a, b, c, and d arranged in order in the horizontal and vertical directions, using an formula, (1+2xcex1)xc3x97(Sb+Sc)/2xe2x88x92Saxc3x97xcex1xe2x88x92Sdxc3x97xcex1 that is rewritten from an operation formula using the coefficient xcex1 and the signal level of each pixel, 0.5xc3x97(Sb+Sc)+xcex1xc3x97(Sbxe2x88x92Sa)+xcex1xc3x97(Scxe2x88x92Sd), and reduces the 4-pixels blocks into 3-pixels blocks, each consisting of pixels a, B, and d arranged in order in the horizontal and vertical directions.
With this invention, correlation between the pixel to be thinned out and the pixel adjacent thereto are detected to produce coefficient according to the correlation, and corrected values for the signal levels of the pixels are calculated using a predetermined operation formula with less calculation quantity in the production of a thinned-out binary image or natural image. Therefore, a sharp image close to its original can be displayed with a simple and small circuitry.
In the image processing device of the present coefficient xcex1 output from the coefficient determining means is determined by the detection of the difference between two signals or gray levels of the pixel to be thinned out and the pixel adjacent thereto. When the difference in signal or gray level between two pixels is larger than a value predetermined as a discrimination reference value, predetermined coefficient xcex1 is output. When the difference in signal or gray level is smaller than the value predetermined as the discrimination reference value, coefficient xcex1 is output as 0. The device can provide a sharp image close to its original in the production of a thinned-out binary image.
In the image processing device of the present invention the coefficient xcex1 output from the coefficient determining means is determined by the detection of the difference between two signal levels or gray levels of the pixel to be thinned out and the pixel adjacent thereto, and the difference between two signal levels or gray levels of two pixels is ranked by a plurality of values predetermined as discrimination reference values so that a predetermined coefficient xcex1 can be selected from a plurality of values and supplied. The device can provide a sharp image close to its original in the production of a thinned-out natural image.
Moreover, in the image processing device of the present invention the thinning-out pulse generating means supplies thinning-out pulses having a width of one pixel or one line to the operation means at regular intervals according to an arbitrary reduction ratio in the horizontal or vertical direction of the image responsive to horizontal and vertical synchronizing signal and pixel clock signal that are input signal. Since the device performs thinning-out processing by generating thinning-out pulses and instructing pixels to be thinned out using a simple structure in the production of a thinned-out image, it can provide a sharp image close to its original.
The image processing device of the present invention includes delay means for delaying an image by every one pixel in a horizontal direction; pulse generating means for generating thinning-out pulses according to a reduction ratio in the horizontal direction; coefficient determining means for detecting correlation between pixels to be thinned out instructed by the thinning-out pulse generating means and pixels adjacent thereto in a plurality of successive pixels obtained by the delay means, and for determining coefficients according the correlation; and operation means for performing computations on the pixels to be thinned out and the pixels adjacent thereto according to the coefficients, and is characterized by producing image data using a predetermined operation formula in the operation means and reducing the image by every one pixel in the horizontal direction.
The image processing device of the present invention includes delay means for delaying the image by every one line in a vertical direction; pulse generating means for generating thinning-out pulses according to a reduction ratio in the vertical direction; coefficient determining means for detecting correlation between the pixels to be thinned out instructed by the thinning-out pulse generating means and the pixels adjacent thereto in a plurality of successive pixels obtained by the delay means, and for determining coefficients according to the correlation; and operation means for performing computations on the pixels to be thinned out and the pixels adjacent thereto according to the coefficients. The device produces image data using the predetermined operation formula in the operation means and reducing the image by every one line in the horizontal direction. Since the correlation between the pixel to be thinned out and the pixel adjacent thereto are detected, and thinning-out processing is performed using coefficients according to the correlation to correct the signal levels of the pixels produced from the pixels to be thinned out and the pixels adjacent thereto, an image processing device capable of displaying a sharp image close to its original can be provided.
In addition, the image processing method of the present invention includes the steps of instructing pixels to be thinned out; detecting correlation between pixels instructed to thin out and pixels adjacent to the pixels instructed to thin out; determining coefficients according to the detected correlation; and producing corrected pixel data using a predetermined operation formula according to data of the pixels instructed to thin out, data of the pixels adjacent to the pixels instructed to thin out, and the determined coefficients. Since this thinning-out processing is performed with corrected signal levels even in binary and natural images, a sharp image close to its original can be provided.
In accordance with the present invention, the image processing device capable of displaying a sharp image closer to its original than conventional devices can be provided because the thinning-out processing is performed with corrected signal levels even in binary and natural images.